{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 目录\n",
    "0. [任务](#任务)\n",
    "1. [创建训练数据](#创建训练数据)\n",
    "    1. [文本编码成数值](#文本转换成数值)\n",
    "    2. [生成直接传入模型的数据](#生成直接传入模型的数据)\n",
    "        - [操作tf.Dataset对象](#操作tf.Dataset对象)\n",
    "        - [编码数据转化为tf.Dataset对象](#编码数据转化为tf.Dataset对象)\n",
    "2. [创建和训练模型(stateless RNN)](#创建和训练模型)  \n",
    "3. [利用模型生成文本](#利用模型生成文本)\n",
    "4. [Stateful RNN](#Stateful-RNN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow import keras\n",
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [任务](#目录)\n",
    "给定语料，训练一个 RNN 模型，该模型可以预测一个句子的下一个字符。该模型就可以通过每次产生一个字符，来生成全新的文本。示例使用莎士比亚的著作作为语料。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [创建训练数据](#目录)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "shakespeare_url = \"https://homl.info/shakespeare\"\n",
    "filepath = keras.utils.get_file('shakespeare.txt', shakespeare_url)\n",
    "with open(filepath) as f:\n",
    "    shakespeare_text = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First Citizen:\n",
      "Before we proceed any further, hear me speak.\n",
      "\n",
      "All:\n",
      "Speak, speak.\n",
      "\n",
      "First Citizen:\n",
      "You are all resolved rather to die than to famish?\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(shakespeare_text[:148])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"\\n !$&',-.3:;?abcdefghijklmnopqrstuvwxyz\""
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"\".join(sorted(set(shakespeare_text.lower())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [文本转换成数值](#目录)\n",
    "将字符编码成整数，将字符串序列编码成整数列表\n",
    "- keras 的`Tokenizer`实现上述编码，但起始索引为 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = keras.preprocessing.text.Tokenizer(char_level=True)\n",
    "tokenizer.fit_on_texts([shakespeare_text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[20, 6, 9, 8, 3]]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.texts_to_sequences(['First'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[11], [1]]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.texts_to_sequences(['\\n', ' '])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['f i r s t']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.sequences_to_texts([[20, 6, 9, 8, 3]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "39"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "max_id = len(tokenizer.word_index)\n",
    "max_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([19,  5,  8, ..., 20, 26, 10])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[encoded] = np.array(tokenizer.texts_to_sequences([shakespeare_text]))-1\n",
    "encoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1115394"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(encoded)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f i r s t   c i t i z e n : \n",
      " b e f o r e   w e   p r o c e e d   a n y   f u r t h e r ,   h e a r   m e   s p e a k . \n",
      " \n",
      " a l l : \n",
      " s p e a k ,   s p e a k . \n",
      " \n",
      " f i r s t   c i t i z e n : \n",
      " y o u   a r e   a l l   r e s o l v e d   r a t h e r   t o   d i e   t h a n   t o   f a m i s h ? \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.sequences_to_texts([encoded[:148]+1])[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [生成直接传入模型的数据](#目录)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "操作tf.Dataset对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(42)\n",
    "tf.random.set_seed(42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = tf.data.Dataset.from_tensor_slices(tf.range(15))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(0, shape=(), dtype=int32) 0\n",
      "tf.Tensor(1, shape=(), dtype=int32) 1\n",
      "tf.Tensor(2, shape=(), dtype=int32) 2\n",
      "tf.Tensor(3, shape=(), dtype=int32) 3\n",
      "tf.Tensor(4, shape=(), dtype=int32) 4\n",
      "tf.Tensor(5, shape=(), dtype=int32) 5\n",
      "tf.Tensor(6, shape=(), dtype=int32) 6\n",
      "tf.Tensor(7, shape=(), dtype=int32) 7\n",
      "tf.Tensor(8, shape=(), dtype=int32) 8\n",
      "tf.Tensor(9, shape=(), dtype=int32) 9\n",
      "tf.Tensor(10, shape=(), dtype=int32) 10\n",
      "tf.Tensor(11, shape=(), dtype=int32) 11\n",
      "tf.Tensor(12, shape=(), dtype=int32) 12\n",
      "tf.Tensor(13, shape=(), dtype=int32) 13\n",
      "tf.Tensor(14, shape=(), dtype=int32) 14\n"
     ]
    }
   ],
   "source": [
    "for x in test_data:\n",
    "    print(x,',', x.numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<_VariantDataset shapes: (), types: tf.int32>\n",
      "<_VariantDataset shapes: (), types: tf.int32>\n",
      "<_VariantDataset shapes: (), types: tf.int32>\n",
      "<_VariantDataset shapes: (), types: tf.int32>\n",
      "<_VariantDataset shapes: (), types: tf.int32>\n",
      "<_VariantDataset shapes: (), types: tf.int32>\n"
     ]
    }
   ],
   "source": [
    "n_steps = 5\n",
    "test_data = test_data.window(n_steps,shift=2, drop_remainder=True)\n",
    "for x in test_data:\n",
    "    print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([0 1 2 3 4], shape=(5,), dtype=int32)\n",
      "tf.Tensor([2 3 4 5 6], shape=(5,), dtype=int32)\n",
      "tf.Tensor([4 5 6 7 8], shape=(5,), dtype=int32)\n",
      "tf.Tensor([ 6  7  8  9 10], shape=(5,), dtype=int32)\n",
      "tf.Tensor([ 8  9 10 11 12], shape=(5,), dtype=int32)\n",
      "tf.Tensor([10 11 12 13 14], shape=(5,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "test_data = test_data.flat_map(lambda window: window.batch(n_steps))\n",
    "for x in test_data:\n",
    "    print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([6 7 8 9], shape=(4,), dtype=int32) tf.Tensor([ 7  8  9 10], shape=(4,), dtype=int32)\n",
      "tf.Tensor([2 3 4 5], shape=(4,), dtype=int32) tf.Tensor([3 4 5 6], shape=(4,), dtype=int32)\n",
      "tf.Tensor([4 5 6 7], shape=(4,), dtype=int32) tf.Tensor([5 6 7 8], shape=(4,), dtype=int32)\n",
      "tf.Tensor([0 1 2 3], shape=(4,), dtype=int32) tf.Tensor([1 2 3 4], shape=(4,), dtype=int32)\n",
      "tf.Tensor([ 8  9 10 11], shape=(4,), dtype=int32) tf.Tensor([ 9 10 11 12], shape=(4,), dtype=int32)\n",
      "tf.Tensor([10 11 12 13], shape=(4,), dtype=int32) tf.Tensor([11 12 13 14], shape=(4,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "test_data = test_data.shuffle(10).map(lambda window: (window[:-1], window[1:]))\n",
    "for x, y in test_data:\n",
    "    print(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "------------------------------------------------------------ Batch 0 \n",
      "X_batch\n",
      "[[ 4  5  6  7]\n",
      " [ 0  1  2  3]\n",
      " [ 8  9 10 11]]\n",
      "==================== \n",
      "Y_batch\n",
      "[[ 5  6  7  8]\n",
      " [ 1  2  3  4]\n",
      " [ 9 10 11 12]]\n",
      "------------------------------------------------------------ Batch 1 \n",
      "X_batch\n",
      "[[10 11 12 13]\n",
      " [ 2  3  4  5]\n",
      " [ 6  7  8  9]]\n",
      "==================== \n",
      "Y_batch\n",
      "[[11 12 13 14]\n",
      " [ 3  4  5  6]\n",
      " [ 7  8  9 10]]\n"
     ]
    }
   ],
   "source": [
    "test_data = test_data.batch(3).prefetch(1)\n",
    "for index, (X_batch, Y_batch) in enumerate(test_data):\n",
    "    print(\"-\" * 60, \"Batch\", index, \"\\nX_batch\")\n",
    "    print(X_batch.numpy())\n",
    "    print(\"=\" * 20, \"\\nY_batch\")\n",
    "    print(Y_batch.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "编码数据转化为tf.Dataset对象\n",
    "- 训练集、验证集、测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<TensorSliceDataset shapes: (), types: tf.int32>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_size = len(encoded)*50//100\n",
    "dataset = tf.data.Dataset.from_tensor_slices(encoded[:train_size])\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_steps = 100\n",
    "window_length = n_steps + 1\n",
    "dataset = dataset.repeat().window(window_length, shift=1, drop_remainder=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<FlatMapDataset shapes: (None,), types: tf.int32>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = dataset.flat_map(lambda window:window.batch(window_length))\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<MapDataset shapes: ((None, None), (None, None)), types: (tf.int32, tf.int32)>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 32\n",
    "dataset = dataset.shuffle(10000).batch(batch_size)\n",
    "dataset = dataset.map(lambda windows: (windows[:,:-1], windows[:,1:]))\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dataset.map(lambda X_batch,Y_batch:(tf.one_hot(X_batch, depth=max_id), Y_batch))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = dataset.prefetch(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32, 100, 39) (32, 100)\n"
     ]
    }
   ],
   "source": [
    "for X_batch, Y_batch in dataset.take(1):\n",
    "    print(X_batch.shape, Y_batch.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [创建和训练模型](#目录)\n",
    "- stateless RNN\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train for 17428 steps\n",
      "Epoch 1/10\n",
      " 1673/17428 [=>............................] - ETA: 39:44 - loss: 2.3181"
     ]
    }
   ],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.GRU(128, return_sequences=True, input_shape=[None, max_id], \n",
    "                     dropout=0.2, recurrent_dropout=0.2\n",
    "                    ),\n",
    "    \n",
    "#     双层太耗时\n",
    "    keras.layers.GRU(128, return_sequences=True, \n",
    "                     dropout=0.2, recurrent_dropout=0.2\n",
    "                    ),\n",
    "    \n",
    "    keras.layers.TimeDistributed(keras.layers.Dense(max_id, activation='softmax'))\n",
    "])\n",
    "model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')\n",
    "history = model.fit(dataset, epochs=10, steps_per_epoch=train_size//batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [利用模型生成文本](#目录)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def preprocess(texts):\n",
    "    X = np.array(tokenizer.texts_to_sequences(texts)) - 1\n",
    "    return tf.one_hot(X, max_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_new = preprocess(['How are yo'])\n",
    "Y_pred = model.predic_classes(X_new)\n",
    "tokenizer.sequences_to_texts(Y_pred+1)[0][-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def next_char(text, temperature=1):\n",
    "    X_new = preprocess([text])\n",
    "    y_proba = model.predict(X_new)[0, -1:, :]\n",
    "    rescaled_logits = tf.math.log(y_proba) / temperature\n",
    "    char_id = tf.random.categorical(rescaled_logits, num_samples=1) + 1\n",
    "    return tokenizer.sequences_to_texts(char_id.numpy())[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def complete_text(text, n_chars=50, temperature=1):\n",
    "    for _ in range(n_chars):\n",
    "        text += next_char(text, temperature)\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_new = preprocess([\"How are yo\"])\n",
    "Y_pred = model.predict_classes(X_new)\n",
    "tokenizer.sequences_to_texts(Y_pred + 1)[0][-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 模型会生成下一个字符的概率分布，每次选择最大概率的字符可能导致同样的单词不断重复\n",
    "- 根据某个温度值对概率分布进行重新加权\n",
    "- 根据重新加权后的分布对下一个字符进行随机采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def next_char(text, temperature=1):\n",
    "    X_new = preprocess([text])\n",
    "    y_proba = model.predict(X_new)[0, -1:, :]\n",
    "    rescaled_logits = tf.math.log(y_proba) / temperature\n",
    "    char_id = tf.random.categorical(rescaled_logits, num_samples=1) + 1\n",
    "    return tokenizer.sequences_to_texts(char_id.numpy())[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def complete_text(text, n_chars=50, temperature=1):\n",
    "    for _ in range(n_chars):\n",
    "    text += next_char(text, temperature)\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(complete_text(\"t\", temperature=0.2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(complete_text(\"w\", temperature=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [Stateful RNN](#目录)\n",
    "- 上述模型在每个训练批次时，隐藏状态初始化为 0，每个时间步更新隐藏状态参数，最后一个时间步丢弃该参数；\n",
    "- 将不同批次数据进行训练时，隐藏状态初始值为上一次训练的最终隐藏转态；即为 stateful RNN \n",
    "- 对应的传入模型的数据需要相应的更改，输入序列不能由重叠\n",
    "- 因此可以学习序列较长期的模式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = tf.data.Dataset.from_tensor_slices(encoded[:train_size])\n",
    "dataset = dataset.window(window_length, shift=n_steps, drop_remainder=True)\n",
    "dataset = dataset.flat_map(lambda window: window.batch(window_length))\n",
    "dataset = dataset.batch(1)\n",
    "dataset = dataset.map(lambda windows: (windows[:, :-1], windows[:, 1:]))\n",
    "dataset = dataset.map(\n",
    "lambda X_batch, Y_batch: (tf.one_hot(X_batch, depth=max_id), Y_batch))\n",
    "dataset = dataset.prefetch(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    keras.layers.GRU(128, return_sequences=True, stateful=True,\n",
    "                     dropout=0.2, recurrent_dropout=0.2,\n",
    "                     batch_input_shape=[batch_size, None, max_id]),\n",
    "    keras.layers.GRU(128, return_sequences=True, stateful=True,\n",
    "                     dropout=0.2, recurrent_dropout=0.2),\n",
    "    keras.layers.TimeDistributed(keras.layers.Dense(max_id,activation=\"softmax\"))\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 但是在不同 epoch 时，模型的状态应该重置\n",
    "    - 可以使用模型的 callbacks 参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ResetStatesCallback(keras.callbacks.Callback):\n",
    "    def on_epoch_begin(self, epoch, logs):\n",
    "        self.model.reset_states()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"sparse_categorical_crossentropy\", optimizer=\"adam\")\n",
    "model.fit(dataset, epochs=50, callbacks=[ResetStatesCallback()])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
